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Abstract 

Continuous-time discrete-state random Markov chains generated by 
a random linear differential equation with a random tridiagonal matrix 
are shown to have a random attractor consisting of singleton subsets, es- 
sentially a random path, in the simplex of probability vectors. The proof 
uses comparison theorems for Caratheodory random differential equations 
and the fact that the linear cocycle generated by the Markov chain is a 
uniformly contractive mapping of the positive cone into itself with respect 
to the Hilbert projective metric. It does not involve probabilistic proper- 
ties of the sample path and is thus equally valid in the nonautonomous 
deterministic context of Markov chains with, say, periodically varying 
transitions probabilities, in which case the attractor is a periodic path. 
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1 Introduction 



The simplex Eat in K defined by 

Sat = I P = (pi, ■ • ■ ,Pat) T : = 1, Pi,--- ,Pn £ [0, 1] L 

is invariant under the dynamics of the system of ordinary differential equa- 
tions 

in K , where the TV x TV matrix Q satisfies 

ljvQ = 0. 

Here In is the column vector in with all components equal to 1. 

The system fl} restricted to Ejv generates a continuous-time finite- 
state Markov chain with a generator Q. Such Markov chains are common 
in the biological sciences with generators given by an TV x TV tridiagonal 
matrix 
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927V-5 — (927V-4 + 927V-3) Q2N-2 
O 92JV-3 —Q2N-2 



(2) 



with positive off-diagonal components {qt}, see [U [2] [3] . The components 
of Q may depend on time or be random, or both, i.e., depend on a stochas- 
tic process, in which case Q will be written Q(9toS) (the undefined terms 
will be explained later). 

The asymptotic behaviour of the discrete-time version of such Markov 
chains were investigated by the authors in .4.. Their transition matrices 
are of the form In + Q(t n )A n , where In is the TV x TV identity matrix, 
Q(t n ) is value of the generator matrix Q at time t n , i.e., Q(t n ) = Q(8 tn u>), 
and A n is a positive parameter. This is just the Euler scheme for the 
differential equation on S/? with the time step A„. It was shown 
in [3] that these discrete-time Markov chains generate a linear random 
dynamical system which has a random attractor consisting of singleton 
components sets, essentially a random process of probability vectors in 
Eat. 

In this paper the corresponding result will be established for contin- 
uous-time finite-state Markov chains generated by the differential equation 
JTJ in Ejv, where Q is an TV x TV tridiagonal matrix of the form @ with 
components driven by a stochastic process. This will be proved directly. 
An indirect proof using the convergence of the random attractors of the 
Euler scheme requires them to have a uniform rate of attraction. This is 
the inverse problem of what is usually discussed in numerical dynamics, 
where it is shown that the numerical scheme preserves the properties of 
continuous-time systems generated by the differential equation, see [S] E] - 

The paper is structured as follows. Basic properties of differential 
equations satisfying the Caratheodory property are recalled in Section [2] 
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In particular, in Subsection 12.11 properties of deterministic linear differ- 
ential equations of the type (JXJ) are investigated, while in Subsection 12.21 
some notions on related random differential equations (RDE) of the type 
((TJ) are recalled. Finally, in Subsection l2.3l it is shown that that the related 
RDE preserves the positivity of initial values and in Section the main 
result of the paper is presented, a theorem on the existence of a random 
attractor consisting of singleton component subsets in the system under 
consideration. 

2 Caratheodory differential equations 

The linear ordinary differential equation {TJ with the matrix Q depending 
on a stochastic process is a differential equation of the Caratheodory type. 
Some basic properties of such equations are recalled here along with some 
comparison theorems, which will be used to establish the positivity of 
solutions of the resulting random differential equation. 
An ordinary differential equation on M. N , 

§ = /(*,-), 0) 

is called a Caratheodory differential equations if the vector-field f(t, x) 
satisfies the Caratheodory conditions: 

CI: for every fixed t, the function x H > f(t,x) is continuous; 

C2: for every fixed x, the function t i-> f(t,x) is measurable in t and 
there exist a function m^it), which is Lebesgue integrable on every 
bounded subinterval, such that \\f(t, x)|| < m a .(t). 

A solution of differential equation Q is a vector- valued function x(t) 
which is absolutely continuous on some interval and satisfies (J3j almost 
everywhere on this interval. The existence of solutions of ((3} on the whole 
real axis R as well as their uniqueness follows from the Caratheodory 
conditions, if, in addition, the function f(t, x) is defined for all (t, x) £ 
R 1 x R^ and satisfies a local Lipschitz condition in a; in a neighbourhood 
of every initial point (to,xo). 

A solution x(t) — (xi(t),X2(t), . . . ,XN(t)) T of ((3J is called positive 
(strongly positive) if 

Xi(t) > (> 0) for all t and i = 1, 2, . . . , N. 

The next condition guarantees the positivity of solutions of $Jfy, see, 
e.g., [3 H H [10]. A function / = (/i,/ 2 , . . . , /jv) t :l 1 xl f, ^l A ' is 
called quasipositive, or off-diagonal positive, if, for each i — 1, 2, . . . , N, 

fi(t,xi, ■ ■ ■ ,Xi-i,0, Xi+i,. ..,xn) > 0, 

whenever Xj > for j ^ i. A function / = (/i, /2, ■ • ■ , /jv) t is called 
strongly quasipositive, or strongly off-diagonal positive, if, for each i = 
1,2,. ..,N, 

fi(t,x\, . . . ,Xi-i,0, Xi+i, x N ) > 0, 
whenever Xj > for all j and . Xj > 0. 
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The proof of the following theorem for the case that f(t, x) is continu- 
ous in t and x can be found, e.g., in [FJ Lemma 4.1]. It was noted in [7] that 
similar statements are also valid when f(t,x) satisfies the Caratheodory 
conditions. 

Theorem 1 If the vector-field f of ((3| is quasipositive, then the solution 
x(t) of ([3| satisfying the initial condition x(0) = xo = (2:0,1, 2:0,2, • ■ • , xo,n) 
is positive for t > whenever xo,i > for i — 1,2, . . . , N . 

If the vector-field f of ^ is strongly quasipositive, then the solution 
x(t) of (0) satisfying the initial condition x(0) = xq = (2:0,1, 2:0,2, • ■ • , xq,n) 
is strongly positive for t > whenever xo : i > for i — 1,2, . . . , TV . 

2.1 Deterministic linear differential equations 

Unfortunately the requirement that a vector-field function be quasipos- 
itive is rather restrictive and does not apply to a deterministic linear 
differential equation 

f =Q P , P eR N (4) 

with a tridiagonal matrix Q given by ©. 

There is, however, another useful notion that can be used in combina- 
tion with it. An N x TV matrix Q with non-negative off-diagonal elements 
is said to have a path of nonsingularity [8j if there is a set of indices 
with i n = ii such that all the elements qi-i j+1 axe strictly 
positive and this set of indices contains all the numbers 1, 2, . . ., TV. In 
particular, the matrix Q defined by ((2| has the path of nonsingularity: 

{ir, i 2 , . . . , i 2N -i} = {1, 2, . . . , TV - 1, TV, TV - 1, . . . , 2, 1}. 

Recall that a set K in a Banach space is called a cone if it is convex, 
closed with tK C K for any real t > and K n —K = {0}, see, e.g. 
[11] . Fix a norm || ■ || in and denote by the cone of elements 

o 

x — (xi, X2, ■ ■ ■ , xn) T S with nonnegative components and by 
the interior of , which is clearly non-empty. In addition, denote by 
the unit ball in the norm || ■ ||. 

Theorem 2 Suppose that the matrix Q in Q with nonnegative off-dia- 
gonal entries has a path of nonsingularity. Then, for any non-zero initial 
condition p(0) = p £ K+, the solution p(t,p ) is strongly positive for all 
t > 0. 

Moreover, for every bounded interval [Ti,T2] C (0, 00), there is a num- 
ber a(Ti, T 2 ) > such that 

p(t;p ) + a(Ti,Ta)|[po || C (5) 

for all p e \ {0} and t e [Ti,T 2 ]. 

Proof. The first part of Theorem [2] follows from [8j Theorem 4.7], while 
the second part is clear modulo the first part. 

Remark 1 The inclusion ([5| means that the solution operator p t->- 

o 

p(t,p ) maps the set \ {0} into its interior for every t > 0. 
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Remark 2 If the matrix Q in Q satisfies the conditions of Theorem 
as well as 1%Q = 0, then p(t;Ejv) Q Sjv /or a// i > 0. Moreover, the 

o 

simplex Ejv is mapped by p(t; •) into its interior Sjv /or ewn/ t > 0. 

2.2 Random linear differential equations 

Let (Q, J-, P) be a probability space. In the theory of random dynamical 
systems [121 113] the noise process that drives the dynamics is given in 
terms of an abstract metric dynamical system. 

Definition 1 A metric dynamical system (MDS) 6 = {#t} t6K on a prob- 
ability space (f2, J- ', P) is a family of transformations 9 t : — > fi, t G R, 

1. it is one-parameter group, i.e., 

6o = ida, t o 8 S — 8 t +s for all t, s G R; 

2. (t,Cc») i — y dtui is jointly smeasurable; 

3. 0<P = P /or a// t 6 R, i.e., P(0 t _B) = P(B) /or all B £ F and t G R. 
Consider the random linear differential equation [12H13| 

^ = Q(e t oj)p, p(0)=p GR Ar , (6) 

driven by the metric dynamical system 9 on a probability space (Q, T ', P), 
where the coefficients oj : £1 — > R in the AT x A^-matrix Q are locally 
bounded ^-measurable functions. 

More precisely, assume that there exists a random variable C with 
C(oj) > such that 

ct + l 

C{6 t u) dt < oo for all a G R, (7) 
and 

\qi(u)\ < C(u) for i = 1,2, . . . , 2JV - 2 (8) 
for all w 6 !!. It is to be emphasized that assumptions J7) and (0 are 
stated here for all ui G Q. As explained in [131 p. 56] this does not restrict 
generality. 

Remark 3 Conditions and JS} are satisfied if 
\qi(u)\ < C* /or a// i ana" cj G 
/or some constant C* > 0. 

The random differential equations is thus a Caratheodory ordinary 
differential equation for each sample path. For each w 6 f! the vector- 
field mapping f(t,p) = Q(9 t uj)p satisfies the Caratheodory conditions 
and (0 and is linear and thus globally Lipschitz in p. It follows that the 
initial value problem has a unique global solution p(t,u) = p(t,ui;p Q ) 
for any initial value p G R^, where this solution exists for all t > 
(see [13] for more details explicitly in terms of the random differential 
equations). Moreover, this solution depends continuously on p for each 
cj G SI. 

It will now be shown that RDE (0 preserves the positivity of initial 
values. 
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2.3 Positivity of solutions 



The results from the previous sections will now be applied to the random 
linear differential equation (RDE) in W N 



dp 



= Q(9 t u) P , P (0) = Po e 



(9) 



driven by a metric dynamical system 0, where the matrix Q(uo) is of the 
tridiagonal form ([2]) and satisfies 



for all wen. 



(10) 



Suppose further that the coefficients in the matrix Q(uj) are locally 
bounded ^-measurable functions satisfying uniform bounds 



< c„ < qi{u) < C, < oo, i = 1,2, 



,2N -2, wefl. 



(11) 



Then, by Remark[3] Q(uj) satisfies the conditions JJJ and ((§]). 
Define (a deterministic) tridiagonal matrix 



9i 92 
9l 92 



O 



94 



o 



with the off-diagonal elements qt 



92AT-5 9JV-1 92iV-2 
92iV-3 9jv 



gi = inf gi(w), i = 1, 2, . 
and the diagonal elements q~i satisfying 



,2N- 



qi = inf {-9i (w)}, 

£i = inf {-(#21-2(0;) + 52i-i(w))}, i = 2, 
qat = inf {-<?2iv-2(o;)}. 



,2N ■ 



Then 

Q(w) > Q, for all w € fl, 

where the inequality between the matrices is interpreted componentwise. 
The off-diagonal elements q~i of the matrix Q are strictly positive since 



0<c»<9i, i = 1, 2, 
Consider the differential equation 



,2.2V- 



(12) 



(13) 



and denote the solution of this equation with the initial condition q(0) 
q €R N by q(t;g Q ). 
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Remark 4 It follows from (|12[l that Q is a matrix with non-negative off- 
diagonal elements that has a path of nonsingularity. Then by Theorem [J] 
and Remark[J\the solution operator q(t; •) maps the set \ {0} into its 

o 

interior for each t > maps. Moreover, relations of the type ([5j> hold 
for it with appropriate parameters. 

Remark 5 The relation 1%Q = is not valid in general. Then the sim- 
plex "En will not be invariant under q(t; •), but this will be of no importance 
in what follows. 

Theorem 3 Let 6 a metric dynamical system and let Q(uj) be a matrix of 
the tridiagonal form which satisfies (| 10f) and Then, the solution 

p(t;u>,p ) of the random linear differential equation ([9]) satisfies 

p(t;uj,p ) > q(t;p ) for all uj G Q, t> 0,p G K+, (14) 

where the inequality is meant componentwise. 

Proof. Fix an w 6 !] and fix the initial conditions p(0) = q(0) = p G 
for the differential equations @ and l)13p . respectively. Then the 
function 

x(t) := p(t;u,p ) - q(t;p ) 
satisfies the differential equation 

^ = Q(8tia)p -Qq = Q{e t io)x + (Q{6 t u) - Q) q 
with the initial condition cc(0) = 0. Denoting 

w{t) := (Q(9 t oj)-Q)q(t;p ), 
this differential equation can be written as 
dT 

— = Q(8 t uj)x + w(t), a3(0) =0. (15) 

The matrix (Q(0tw) — Q) has, by definition, non-negative components 
for every t and lo. By Remark [4] the function q(t;p ) with p G has 
non-negative components for all t > 0. Hence the function w(t) has non- 
negative components. Moreover, the matrix Q{ui) as a matrix with non- 
negative off-diagonal components has a path of nonsingularity, so it follows 
that the differential equation (|15[) satisfies all the conditions of Theorem[T] 
Hence, the solution function x(t) is positive, i.e., its components are non- 
negative. 

Theorem [3] has some important consequences. 
Corollary 1 There following statements are valid: 

(i) for any uj G O. and t > 0, the solution operator p(t\uj,-) maps the 

o 

set K+ \ {0} into its interior K^f; 

(ii) for any cj G and t > 0, the solution operator p(t;uj, •) maps the 

o 

simplex Ejv into its interior Ejv; 
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(iii) for any bounded interval [Ti,T2] C (0, oo) there is a number /3 '(Ti^T?) 
> such that 

P (t;cj,p ) + P(T u T 2 )\\p \\ (s^nSiv) CEat, 

for all ujen, p £ K+ \ {0}, t £ [Ti, T 2 ]. 

To prove Corollary [1] it suffices to note that Assertion (i) follows im- 
mediately from inequality (|14p and Remark [T] This, due to (|10l) . implies 
Assertion (ii). Both these Assertions together with the inclusion ([5]) valid 
for the solution operator q(t; ■) imply Assertion (iii). 

3 Random dynamical systems 

Let (SI, T , P) be a probability space. A random dynamical system [12LI13] 
has a skew product structure consisting of a metric dynamical system 
that models the driving noise process and a cocycle mapping. 

Let M be a complete metric space equipped with the metric p and let 
9 be a metric dynamical system. 

Definition 2 A map ip : R + x Q x M — > M is called a cocycle on M with 
respect to the driving system 9 if it satisfies the initial condition 

<p(0, oj, •) = idx for all lu 6 O, 

and the cocycle property 

tp(t + s, ui, x) = cp(s, 6t0J, ip(t, U3, a;)) for all x £ M, u> £ Q, t,s£M. + . 

The RDE ([6]) generates a random dynamical system (9, ip) on with 
the cocycle mapping ip defined by 

(p(t,uj,p ) — p(t,uj;p ) for all t > 0, uj £ O,, p £ 9. N . 

In fact, the mapping (t,p) h-> ip(t,u),p) is continuous for each u; £ f2, while 
the mapping p h-> ip(t,u),p) is linear. 

3.1 Random attractors 

The asymptotic behaviour of a random dynamical system is described by 
random attractors. 

Definition 3 A random attractor of — {A(uj),uj £ Q,} of a random dy- 
namical system (9,<p) is a family of nonempty compact subsets of M such 
that 

1. the setvalued mapping oj \-t A[u>) is J- -measurable. 

2. si/ is tp-invariant, i.e., 

tp {t,u),A(uj)) = A{9 t Lj) 
for every u> £ S7 and t £ R + , 



8 



3. si pullback attracts T -measurable families & — {D(oj),uj G ft} of 
nonempty compact subsets of M , i.e., 



dist (<p (t, 6-tU>, D {0-tu)) , A{uj)) — > as t 
for every lu G ft 

It follows from Theorem [3] and Corollary [1] that the solution opera- 

o 

tor p(t;u, •) of the RDE ([6]) maps the simplex Eat into its interior Eat 
uniformly for t in bounded intervals from (0, oo). 

The Hilbert projective metric pu on will be used. It can be defined 

as 



p H (x,y) = 



In 



max; yi/xi 



v maxi Xi I yi 

for vectors x — (xi,X2, ■ ■ ■ ,xn) T and y = (j/i, i/2, • ■ • , Vn) T in K+. It 
is fact only a semi-metric on K+, but becomes a metric on a projective 

o 

space. Important here is that the interior Ejv of the probability simplex 
Ejv is the complete metric space with the Hilbert projective metric, see, 
e -g- 5 [1H 1141 115] and references therein on the properties of the Hilbert 
projective metric. In the Russian literature the Hilbert projective metric 
is also called the Birkhoff metric. 

The main result of this paper, Theorem[4]below, requires the following 
definitions. 

Definition 4 A cocycle mapping tp : R + x SJ x M -> M, where (M, p) 
is a metric space, is called uniformly dissipative if there exist a number 
Td G R + and a closed bounded set Mo C M such that 

<p(T d ,cj,M)CM , VcjGfi. 

It is called uniformly contractive if there exist a number T c G K + and a 
number X G [0, 1) such that 

p(tp(T c ,uj,x),<p(T c ,u},y)) < \p(x,y), V u G ft, X,y € M. 

Theorem 4 The restriction of p(t;u,-) to the set Eat is a uniformly 
dissipative and uniformly contractive cocycle (with respect to the Hilbert 
projective metric), which has a random attractor = {A(oj),ui G ft} 
such that set A(oj) = {ci(cj)} consists of a single point for each cj G ft. 
Moreover, the random attractor is asymptotically stable, i.e., 

\\p(t, cj,p ) — a(6tcj)\\ — ► as t — > oo 

o 

for all p G Eat and lu G ft. 

Proof. The proof follows by an application of the discrete-time case in 
[4] to, e.g., the time-one mapping of the RDE, i.e., the cocycle mapping at 
integer time values. The uniform dissipativity of the cocycle ensures the 
existence of a random attractor, while the uniform contractivity implies 
that the components subsets are singleton sets. 
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